Motion and gesture recognition by a passive single pixel thermal sensor system

ABSTRACT

Systems and methods for recognizing a gesture made by a warm object are presented. The system includes a thermal sensor configured to generate a low frequency or direct current signal upon receiving thermal energy. A spatially modulating optic is disposed between the thermal sensor and the warm object. The optic is configured to modulate the thermal energy received by the thermal sensor as a function of an orientation of the warm object with respect to the thermal sensor. An electronics unit in communication with the thermal sensor includes a memory and a processor. The processor is configured by the memory to detect a change in the thermal sensor signal and recognize a characteristic of the thermal sensor signal.

FIELD OF THE INVENTION

The present invention relates to thermal sensors, and more particularly,is related to infrared sensing proximity detectors.

BACKGROUND OF THE INVENTION

Motion detectors typically employ passive infrared (IR) sensors thatdetect the heat from moving persons or other warm objects and generatean electrical signal. Such detectors typically include a pyroelectricmaterial and a multi-modulating optics, often referred as Fresnel lens,alternatingly focusing light on two pyroelectric elements. Thepyroelectric elements generate an electrical signal if the incoming heatflux changes over time. The pyroelectric detector thus acts as naturalelectrical high-pass by being sensitive to motions occurring above acertain typical frequency range. Depending on the sensing element size,the cut-off frequency may be as low as 0.4 Hz for large element sizes orhigher for smaller elements. Typically, motions of a person occur in therange of approximately 0.4 Hz to 4 Hz, so elements and signal processingelectronics in proximity detectors are typically tuned to this range.For decades, pyroelectric infrared (IR) detectors were considered theeasiest approach to motion sensing for light switches and alarm unitssince they deliver high signal levels which could be processed by thenavailable analog electronics.

More recently, a single pixel thermal sensor has been developed todetect a frequency response down to steady-state heat flux (directcurrent (DC)) in conjunction with a multi-element modulating opticswhich modulates the signal over the total field-of-view (FOV) of thesensor. Such a device is able to detect motions within a much broaderfrequency range, even down to DC, which corresponds to the detection ofstationary objects (“Presence Detection”) or to higher frequencies,which additionally allows the detection of faster human bodily motions,such as hand waving or other gestures

Previously, gesture recognition techniques have generally been dividedinto imaging and non-imaging techniques. The imaging techniquestypically employ multi-pixel sensors with optics that map the FOV ontothe different pixels, so motions and gestures can be evaluated by meansof image processing methods. While many of the imaging methods evaluatemotion in a two dimensional plane, imaging may also be performed inthree dimensions by including depth information, which can be achievedby a variety of methods such a time-of-flight, stereo images, structuredlight pattern recognition, or others.

Fewer non-imaging gesture recognition techniques have been employed. Onenon-imaging technique utilizes an electric field, in which change isdetected by means of a capacitive detector array in the sensing plane.Another non-imaging method employs the reflection of an infrared beamsent out by an IR light-emitting diode (IR LED). The beam is reflectedoff an object and is detected by one or more photodiodes. Thesenon-imaging methods incorporate multi-pixel or multi-electrode sensors.The non-imaging solutions do not make an image of the scene, since thereis not a defined relationship between a specific FOV segment with acertain detector pixel.

Analysis of frequency patterns in the output of non-imaging sensingdevices is known. For example, inertial sensors, as commonly employed inmobile device for the detection of motion of the device, may havesoftware that looks for frequency and amplitude patterns and for thefingerprint of a certain detected physical motion. As such, the softwarecan determine, for example, if the user carrying the mobile device iswalking, driving in a car or on a train. It is also possible todetermine if the device is lifted up and placed at the ear of a user totake a call, solely by comparing frequency and amplitude pattern of theinertial sensor output signal with those in a library of gesturesignatures. Such pattern recognition software may be self-learning, andthe library can be extended or adjusted by extracting common patternsfrom other behavior of a user. However, the output of non-imagingsensors has been insufficient to recognize multi-dimensional movement orgestures.

Therefore, there is a need in the industry for a gesture recognitionsolution that addresses at least some of the abovementionedshortcomings.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide motion and gesturerecognition by a passive single pixel thermal sensor system. Brieflydescribed, the present invention is directed to a system configured torecognize a gesture made by a warm object, including a thermal sensorconfigured to generate a low frequency and/or direct current signal uponreceiving thermal energy, a spatially modulating optic disposed betweenthe thermal sensor and the warm object configured to modulate thethermal energy received by the thermal sensor as a function of anorientation of the warm object with respect to the thermal sensor, andan electronics unit in communication with the thermal sensor. Theelectronics unit includes a memory and a processor in communication withthe memory. The processor configured by the memory to perform steps ofdetecting a change in the thermal sensor signal, and recognizing acharacteristic in the thermal sensor signal.

A second aspect of the present invention is directed to a method forrecognizing a gesture of a warm object moving in a monitored space. Themethod includes the steps of receiving incident thermal energy at amodulating optics from a field of view of the modulating optics withinthe monitored space, wherein the modulating optics comprises a pluralityof lenses and/or apertures, directing the incident thermal energyreceived by the modulating optics onto a thermal sensing deviceoptically coupled to the modulating optics, producing, with the thermalsensing device, a direct current output signal that is sustained at alevel proportional to an amount of thermal energy being directed to thethermal sensing device by the modulating optics, and providing theoutput signal to an electronics unit in communication with the thermalsensing device. The electronics unit includes a memory and a processorin communication with the memory. The processor configured by the memoryto isolate a characteristic of the signal and compare the characteristicof the signal to a reference characteristic.

Other systems, methods and features of the present invention will be orbecome apparent to one having ordinary skill in the art upon examiningthe following drawings and detailed description. It is intended that allsuch additional systems, methods, and features be included in thisdescription, be within the scope of the present invention and protectedby the accompanying claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a furtherunderstanding of the invention, and are incorporated in and constitute apart of this specification. The drawings illustrate embodiments of theinvention and, together with the description, serve to explain theprincipals of the invention.

FIG. 1 is a schematic, cross-sectional side view of a first embodimentof a gesture detector.

FIG. 2 is a partial top view of an example of the modulateddetector-optics of FIG. 1.

FIG. 3 is a schematic diagram showing an exemplary system forimplementing the functionality of the detector of FIG. 1.

FIG. 4A is a graph of the signal output from a prior art single pixelthermal sensor without optics looking into the sensor viewing area.

FIG. 4B is a graph of the signal output from the single pixel thermalsensor of FIG. 1 including modulated optics looking into the sensorviewing.

FIG. 4C is the signal from FIG. 4B after band pass filtering.

FIG. 5 is an explanatory diagram of a time-domain waveform produced by ahand gesture as detected by the single pixel thermal sensor of FIG. 1.

FIG. 6 is a graph of three modulation patterns used in exemplary opticsused in conjunction with the single pixel thermal sensor of FIG. 1.

FIG. 7 is a flowchart of an exemplary method for recognizing a movementgesture in a monitored space.

DETAILED DESCRIPTION

The following definitions are useful for interpreting terms applied tofeatures of the embodiments disclosed herein, and are meant only todefine elements within the disclosure. No limitations on terms usedwithin the claims are intended, or should be derived, thereby. Termsused within the appended claims should only be limited by theircustomary meaning within the applicable arts.

As used within this disclosure, “lens” refers to an optical element thataffects the amount and/or direction of electromagnetic radiation orlight conveyed through it. A lens may affect transmitted radiation basedon the size and/or geometry of an aperture and the shape and spacing ofradiation transmitting media, for example, glass. As used herein, a lensmay refer to a passive optical element, or an active optical element.

As used within this disclosure, “warm object” refers to an objectemitting heat detectable by a thermal presence detector. A warm objectgenerally refers to a person or animal.

In general, the phrase “monitored space” refers to a physical area(e.g., a room, hallway, outdoor area, etc.) where a presence detector ispositioned and where the detector can potentially detect the warmobject. However, a monitored space may also refer to a smaller region inthe proximity of a thermal imaging device, including at least a portionof a field of view of the detector.

Reference will now be made in detail to embodiments of the presentinvention, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers are used in thedrawings and the description to refer to the same or like parts.

Embodiments of methods and devices for gesture recognition with a singlepixel thermal sensor system using appropriate evaluation and optics arepresented. An exemplary embodiment of a gesture recognition systemincludes a thermal detector with modulating optics. The spatialmodulation of the FOV of the detector enhances the motion signal outputby the detector and may provide a signature which may be recognized, forexample, in a certain frequency band. Such a signature can be evaluatedby either software or appropriate hardware.

FIG. 1 is a schematic, cross-sectional side view of a first embodimentof an exemplary detector 100 configured to recognize a gesture basedupon the detected presence, position, motion and/or direction of motionproduced by a warm object within a monitored space where the detector100 is positioned and where the detector 100 can potentially detect thewarm object.

The detector 100 has a sensor module 102 with a thermal sensing device120, for example a thermopile, and an intensity modulating optics 104 atleast partially covering the sensor module 102. The optics 104 mayconsist of a plurality of lenses, each of which is arranged to directincident thermal energy from the monitored space onto at least part ofthe sensor module 102. In some implementations, each individual lens ofthe modulating optics 104 directs incident thermal energy from one ofmultiple different physical zones in the monitored space onto the sensormodule 102. Such physical zones may be overlapping or non-overlapping,both in terms of angular range in front of the detector 100 and distancefrom the detector 100.

The modulating optics 104 may be directly attached to the detector 100as depicted, or it may also be mounted at a distance to the detector.There may be a cavity 114 within the detector 100 separating the optics104 from the sensor module 102, or the optics 104 may directly abut thesensor module 102. The modulating optics can assume many forms asdescribed below.

The thermal sensing device 120 is generally operable to produce a directcurrent (DC) output that is substantially proportional to an amount ofthermal energy (depicted by dashed arrows) being received at thatthermal sensing device 120. The DC output produced by the thermalsensing device 120 remains generally constant as long as the amount ofthermal energy being delivered to that thermal sensing device 120remains substantially constant. Increases in the amount of thermalenergy being delivered to the thermal sensing device 120 generallyresult in a proportional increase in the DC output being produced by thesensing device 120. Likewise, decreases in the amount of thermal energybeing delivered to the thermal sensing device 120 generally result in aproportional decrease in the DC output being produced by the sensingdevice 120. Under the first embodiment, the thermal sensing device 120is a single pixel thermal sensor. The DC output from the thermal sensingdevice 120 may be either a DC voltage or a DC current.

While the thermal sensor module 102 has a single pixel thermal sensingdevice 120, alternative embodiments may include two or more thermalsensing devices 120, where each thermal sensing device 120 has one ormore pixels. However, the gesture recognition functionality describedbelow may be accomplished on a detector 100 having only one single pixelthermal sensing device 120. In general, a thermopile is an electronicdevice that converts thermal energy into electrical energy. A thermopileis generally composed of several thermocouples electrically connectedusually in series or, less commonly, in parallel, to produce a singledirect current (DC) output.

As noted above, in some implementations, the thermal sensor module 102has multiple thermal sensing devices 120 (e.g., multiple thermopiles).In some implementations, all of the thermal sensing devices in a sensormodule 102 are connected together electrically to produce a single DCoutput signal from the sensor module 102. In some implementations, thethermal sensing devices 120 are configured so as to produce multipledifferent DC output signals from the sensor module 102.

As illustrated in the first embodiment, the sensor module 102 isembedded within the substrate or housing 110 and the modulating optics104 is supported above the sensor module 102 atop optional legs 115 andthe substrate 110. The optics 104 may have a variety of possibleconfigurations. For example, the optics 104 can include a Fresnel lensor other lenses, Fresnel zones, zone plates, holographic opticalelements, diffractive optical elements, refractive optical elements,binary optical elements, simple apertures, and any combination of theseor any other arrangement that provide an intensity modulation with aspatially moving object. The modulating optics 104 may also includeadditional elements, for example, a spatial aperture array with total orpartial light exclusion between apertures, a grating, a coding plate ordisc, or any combination in any suitable arrangement in front of thesensor module 102.

FIG. 2 is a partial top view of the detector 100 in FIG. 1. Theillustrated view shows one exemplary implementation of the modulatingOptics 104 of the detector 100. The function of the optics 104 is todivide the monitored space into different segments. This segmentation isachieved by having optical elements on the modulating optics directingradiation only from a certain segment onto a certain thermal sensingdevice 120 within the module 102. These optical elements may coincidewith discrete physical regions such as in the illustrated view of FIG.2, but may also be distributed over the modulating optics 104 surface asit may be the case by using holographic optical elements, for example.

Each optical element typically not only divides the monitored space intosegments, but also bundles radiation incident from that segment onto aspecific thermal sensing device 120 (FIG. 1). If a warm object, forexample, the hand of a person, moves through a segment, the signalgenerated by the respective thermal sensing device 120 starts at a lowlevel and reaches the maximum when the hand is present at or near themiddle of the segment. If the hand moves further, the signal leveldecreases to a low level again. A hand moving through multiple zoneswill thus generate a changing output pattern with maximum signal beingfully within the segment and minimum signal being at the boundariesbetween segments.

The total number of monitored space segments may be equal or less thanthe number of optical regions of the modulating optics 2 times thenumber of thermal sensing devices 120 within the sensor module 102. Inone embodiment, the modulating optics 104 has alternating regions ofrelatively high transmissivity and relatively low transmissivity. Ingeneral, the relatively high transmissivity regions allow a relativelylarge fraction of incident thermal energy at a wavelength of interest topass through to the sensor module 102, whereas the relatively lowtransmissivity regions allow a relatively small fraction of thermalenergy at the wavelength of interest to pass through to the sensormodule 102. In a further embodiment, as illustrated in FIG. 2, thecentral portions 216 of each lens 214 form regions producing relativelyhigh output signals and the peripheral portions of each lens 214 and thespaces between adjacent lenses 214 form regions with relatively lowoutput signal from the sensing device.

The alternating regions of relatively high output signal and relativelylow output signal help facilitate motion detection, because the fractionof thermal energy from the warm object that reaches the thermal sensormodule 102 beneath the modulating optics 104 will change as that objectmoves through the monitored space, for example, from a space thatcorresponds to the relatively high output signal region of themodulating optics 104 to the relatively low output signal region of themodulating optics 104. In effect, the modulating optics 104 takes theconstant thermal energy of the object and modulates it to form analternating signal at the sensing device 120.

In general, the phase “wavelength of interest” refers to a wavelength orrange of wavelengths to which the thermal sensing devices 120 areresponsive (i.e., whatever wavelengths may affect the DC output from thethermal sensing devices). In a typical implementation, the wavelength ofinterest corresponds to the thermal energy emitted by a warm (living)object. In some implementations, the wavelength of interest is between 4μm and 20 μm.

Referring again to FIG. 1, the illustrated detector 100 has anintegrated circuit 106 that may, in various implementations, form acomputer-based processor, a computer-based memory storage device and/orother circuitry to perform and/or support one or more of thefunctionalities described herein. Electrical conductors, for exampletraces that extend along the upper and/or lower surfaces of thesubstrate 110, vias 108 that extend through the substrate, solder bumps112, etc., are provided to connect the internal electrical components ofthe detector 100, and to connect the detector 100 to externalcomponents.

An exemplary system for executing the functionality described in detailabove may be a computer, an example of which is shown in the schematicdiagram of FIG. 3. It should be noted that the physical layout of theblocks shown in FIG. 3 may be distributed over two or more components,so that, for example, the sensor module 102 may located within thedetector 100, while the processor 302 and/or the memory 306 are locatedremotely from the detector.

The exemplary layout shows a processor 302, a storage device 304, amemory 306 having software 308 stored therein that defines at least partof the abovementioned functionalities, input and output (I/O) devices310 (or peripherals), the sensor module 102, and a local bus, or localinterface 312 allowing for communication across subcomponents of thedetector 100.

The local interface 312 can be, for example, one or more buses or otherwired or wireless connections. The local interface 312 may haveadditional elements, which are omitted for simplicity, such ascontrollers, buffers (caches), drivers, repeaters, and receivers, toenable communications. Further, the local interface 312 may includeaddress, control, and/or data connections to enable appropriatecommunications among the aforementioned subcomponents.

The processor 302 is a hardware device for executing software, such asthe software stored in memory 306, or firmware. The processor 302 can beany custom made or commercially available single core or multi-coreprocessor, a central processing unit (CPU), an auxiliary processor amongseveral processors associated with the detector 100, a semiconductorbased microprocessor (in the form of a microchip or chip set), amacroprocessor, or generally any device for executing software orfirmware instructions. The processor 302 can be integrated, for example,into the integrated circuitry 106 of FIG. 1.

The memory 306 can include any one or combination of volatile memoryelements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM,etc.)) and/or nonvolatile memory elements (e.g., ROM, hard drive, tape,CDROM, etc.) or a network connection to external servers. Moreover, thememory 306 may incorporate electronic, magnetic, optical, and/or othertypes of storage media. Note that the memory 306 can have a distributedarchitecture, where various components are situated remotely from oneanother, but can be accessed by the processor 302. The memory 306 can beintegrated, for example, into the integrated circuitry 106 of FIG. 1.

In general, the software 308 includes instructions that, when executedby the processor 302, cause the processor 302 to perform one or more ofthe functionalities of the detector 100 (FIG. 1) disclosed herein. Thesoftware 308 in the memory 306 may include one or more separateprograms, each of which contains an ordered listing of executableinstructions. The memory 306 may contain an operating system (O/S) 320.The operating system may be operable to control the execution ofprograms within the detector 100 (FIG. 1) and may provide scheduling,input-output control, file and data management, memory management, andcommunication control and related services.

The I/O devices 310 may include interfaces to external devices to allowfor outputting collected data or instructions to various peripheralcomponents. The I/O devices 310 may also facilitate uploading softwareand the like to the detector 100 (FIG. 1).

The sensor module 102 may be, for example, an infrared sensor or anykind of sensor that is responsive to thermal energy. The sensor module102 may include a single element sensor or a sensor array including twoor more sensor elements. A sensor array may include multiple sensorelements within a single enclosure, or may include multiple enclosures,where each enclosure includes two or more sensor elements. The sensormodule 102 may be configured to detect only infrared radiation, or maybe tuned to receive wider bandwidths. The sensor module 102 may furtherinclude voltage regulation and noise reduction components. The sensormodule 102 may convey sensing parameters, for example, ambienttemperature and the temperature of a sensed object, to the processor 302via the local interface 312.

Similarly, for an array sensor, the sensor module 102 may conveyparameters for each individual array element, or may send derivedparameters collated from all of the individual array sensor elements.The sensor module 102 may include an analog to digital converter, forexample, to convert signals between analog and digital formats. Inaddition, the sensor module 102 may be configured to autonomously conveyinformation, for example upon startup and when parameter changes aredetected, or by sending periodic parameter reports. The sensor module102 may be configured to convey parameter information when queried orpolled, for example, by the processor 302.

The storage device 304 can be any type of memory storage device. Ingeneral, the storage device 304 is operable to store any data that willhelp the detector 100 perform one or more of the functionalitiesdisclosed herein. The storage device 304 may be integrated into theintegrated circuitry 106 in FIG. 1.

When the detector 100 (FIG. 1) is in operation, the processor 302executes the software 308 stored in the memory 306, communicates data toand from the memory 306 and storage device 304, and generally controlsoperations of the detector 100 (FIG. 1). It should be noted that in someembodiments, one or more of the elements in the exemplary embodiment maynot be present. Additionally, in some implementations, one or more ofthe elements in the exemplary embodiment may be located external to thedetector 100 (FIG. 1).

The detector 100 (FIG. 1) may be used to provide gesture recognition (asa refined mode of motion and presence detection) through a true singlepixel sensor. Under the first embodiment, gesture recognition may beachieved by analyzing the frequency and amplitude spectra of an outputsignal generated by the sensor module 102 that looks through spatiallymodulating optics 104. In contrast with the prior art which only detectsthe motion or presence of a warm object in the FOV of the detector 100(FIG. 1), the gesture recognition functionality derives additionalinformation, such as the nature of the motion, for example, fast, slow,near, far, direction, or even identifies specific gestures, for examplearm or hand waving, fist making, or the formation of geometricalfigures, such as moving the hand in a circle.

FIG. 4A is a graph of the signal output from a single pixel thermalsensor looking into the FOV of the sensor module 102 without any specialoptics (modulating optics 104). For example, The FOV may be between 50°and 120° and a person passing along this FOV in 1 m or 2 m distance fromthe detector 100 (FIG. 1) generates the signal, where the signalamplitude depends upon the distance between the person and the detector100 (FIG. 1). The signal indicates a first peak 410 when the personpasses approximately 1 m in front of the detector 100 (FIG. 1), and asecond peak 420 when the person passes approximately 2 m in front of thedetector 100 (FIG. 1).

FIG. 4B is a graph of the signal output from the single pixel thermalsensor of FIG. 1 looking into the sensor viewing, in the presence of thesame movement detected in the graph of FIG. 4A, where the detector 100also includes the modulating optics 104 (FIG. 1). With the addition ofthe modulating optics 104, the resulting signal of a moving person ismodulated as it passes through the FOV of the detector 100 (FIG. 1). Themodulating optics 104 (FIG. 1) accentuates the frequency content of theresulting output signal.

FIG. 4C shows the output signal of FIG. 4B after filtering with a(numerical) band pass filter, which translates the motion-based signalmodulation into an output signal. This signal determines the motion andits amplitude level provides information on the distance. FIG. 4C showsexemplary band pass filtered raw data, which illustrates how filteringenhances the detectability. In a prior art motion detector, at thispoint the filtered signal may be integrated and compared to a thresholdvalue deciding whether motion has occurred or not. The band passfiltering operation may be performed, for example, by the processor 302(FIG. 3) as configured by the software 308 (FIG. 3), or by anothercomponent, for example, a dedicated signal processor (not shown) incommunication with the local bus 312 (FIG. 3).

Other signal processing of the signal from the sensor module 102 may beperformed by this system, for example, conversion from the time domainto the frequency domain, for example, via a fast Fourier transform (FFT)processor. Signal filtering, signal smoothing, noise reduction and othersignal processing functions are also possible. As with the band passfiltering, this signal processing may be performed by the processor 302(FIG. 3) as configured by the software 308 (FIG. 3), or by one or morecomponents, for example, a dedicated signal processor or filter.

However, for gesture recognition under the first embodiment, additionalinformation carried in the frequency pattern may be used to identifygestures. Specific motions or sequences of motion may exhibit a certaintime and or frequency pattern in the signal output from the detector 100(FIG. 1). As an example, FIG. 5 shows that a hand making a first has adistinct pattern 510, 520 in the received IR signal 500. The first has asmaller area and thus the signal 500 amplitude shows a sudden drop 520.Such a pattern may be recognized either in the time domain signal or inthe frequency domain. The recognition of such a gesture may be used by agesture recognizing process performed by the processor 302 (FIG. 3) asconfigured by the software 308 (FIG. 3). For example, the gesture may berecognized as a “click” event replacing a computer-mouse function.

Pattern recognition may be performed by the processor 302 (FIG. 3) asconfigured by the software 308 (FIG. 3). For example, the processor 302(FIG. 3) may process the detector (FIG. 1) output signal to isolatecharacteristics in the signal from the sensor module 102 in thefrequency domain and/or the time domain. The processor 302 (FIG. 3) mayidentify correlations between two or more characteristics in thefrequency domain and/or the time domain. The processor may then look tomatch these characteristics and/or correlations with previously storedpatterns, for example, reference patterns stored in the memory 306 (FIG.3), also referred to as signatures. The signature may be stored locallyor remotely, for example, in a remote server with access to a library ofsignatures. A signature may include a single signal characteristic,multiple characteristics, and relationships between characteristics, forexample, in time, amplitude, and/or frequency. Correlation of frequencyand time domain characteristics may be used to determine, for example,the speed of a gesture, or repetitions of a motion.

An exact match between a set of analyzed characteristics and a storedsignature may not be needed for the processor 302 (FIG. 3) to declare amatch. For example, a match may be declared if the correlation betweensignature characteristics and characteristics of the analyzed signal areabove a configurable threshold level. The matching of analyzedcharacteristics and stored signatures may be performed by a signaturelibrary module.

As noted above, the modulating optics 104 may include a combination ofapertures and radiation shaping components. In the case of the simplemotion detection, a specific gesture pattern in the field of view of themodulating optics 104 is enhanced if the optics that project the objectonto the sensor module 102 shows a modulation pattern as a function ofview direction. This modulation can be, for a Fresnel lens, another typeof microlens array, or other signal modulating patterns, such as a lensarray 104 including more or fewer transmitting regions. Such modulationpatterns in the modulating optics 104 cause the sensor module 102 toproduce a signal that can be used for gesture recognition, without theuse of an imaging sensor.

It should be noted, that while a single pixel is sufficient to determinecertain motions or gestures, the use of multi-pixel solutions can beadvantageous in some scenarios, for example, if recognition of thegesture is facilitated by determining the direction of movement. Evenso, it is important to reiterate that such recognition is performedwithout an imaging sensor, since there is no unique relationship betweenan object point and a pixel.

The spatial distance of the modulation pattern with respect to thesensor module 102 combined with the speed of the motion contribute theresulting frequency pattern detected by the sensor 102. Arrangements ofthe modulating optics 104 may be used where the intensity modulation hasseveral different periods of intensity modulation as depicted by thegraph 600 in FIG. 6. The graph 600 depicts regions of hightransmissivity 600 as white blocks, and regions of lower transmissivity670 as hashed blocks. In this example, three different high and lowtransmitting patterns 610, 620, 630 are placed along the x-axis withrespect to the sensor 102, so a movement in the x-direction is modulatedby 3 different frequencies corresponding to the transmitting patterns610, 620, 630. This enhances detectability by the gesture recognitionmodule and provides more details for determining recognizablecharacteristics (fingerprints) for the detection. While the graph 600 issimplified to show only two levels of transmissivity 660, 670, there isno objection to patterns with three or more levels of transmissivity.Similarly, while the graph 600 shows simple, linear patterns, themodulating optics 104 may have more complex transmissivity patterns thatmay or may not be linear.

The use of multimode optics to modulate the thermal radiationcontributes additional information to the output signal of the singlepixel thermal detector, enabling gesture recognition techniques previousunavailable to non-imaging sensors, optical infrared detectors inparticular.

FIG. 7 is a flowchart of an exemplary method for recognizing a movementgesture in a monitored space. It should be noted that any processdescriptions or blocks in flowcharts should be understood asrepresenting modules, segments, portions of code, or steps that includeone or more instructions for implementing specific logical functions inthe process, and alternative implementations are included within thescope of the present invention in which functions may be executed out oforder from that shown or discussed, including substantially concurrentlyor in reverse order, depending on the functionality involved, as wouldbe understood by those reasonably skilled in the art of the presentinvention.

Incident thermal energy is received at a modulating optics 104 (FIG. 1),as shown by block 710. For example, the thermal energy may be receivedfrom a person making a gesture within the FOV of the modulating optics104 (FIG. 1). The modulating optics 104 (FIG. 1) directs the incidentthermal energy received by the modulating optics 104 (FIG. 1) onto athermal sensing device 102 (FIG. 1), as shown by block 720. The thermalsensing device 102 (FIG. 1) produces a direct current output signalproportional to an amount of thermal energy directed to the thermalsensing device 102 (FIG. 1) by the modulating optics 104 (FIG. 1), asshown by block 730.

The modulating optics 104 (FIG. 1) may include regions with differentthermal transmission characteristics, for example causing individualregions to transmit different modulations of the received thermal energyto different or overlapping regions of the thermal sensing device 102(FIG. 1). These various modulations may be indicated in the outputsignal of the thermal sensing device 102 (FIG. 1). The modulating optics104 (FIG. 1) is configured to modulate the thermal energy received bythe thermal sensing device 102 (FIG. 1) as a function of an orientationof the gesturing object with respect to the thermal sensor. Such anorientation may be, for example, distance, angle, rate of motion, and/ordirection of motion.

One or more characteristics of the signal may be isolated, as shown byFIG. 740. For example, the signal may be processed to isolatecharacteristics such as amplitude and time between events in the timedomain, and frequency components in the frequency domain. This isolationof characteristics may be performed by a software controlled processor,a hardware signal processing component, or a combination of the two, asdescribed previously. The one or more characteristics of the signal arecompared to one or more reference characteristics, as shown by FIG. 750.The reference characteristics may be stored in a local memory, forexample, in a reference characteristics library, or may be storedremotely. If the one or more characteristics are sufficiently similar tothe one or more reference characteristics, a gesture associated with theone or more reference characteristics is deemed to be recognized. Thethreshold for determining whether the characteristics are sufficientlysimilar may be controllable. For example, level thresholds may beestablished where a certain number of characteristics, or a certainsimilarity of characteristics must be matched before a gesture isdetermined to be recognized.

In summary, it will be apparent to those skilled in the art that variousmodifications and variations can be made to the structure of the presentinvention without departing from the scope or spirit of the invention.In view of the foregoing, it is intended that the present inventioncover modifications and variations of this invention provided they fallwithin the scope of the following claims and their equivalents.

What is claimed is:
 1. A system configured to recognize a gesture madeby a warm object, comprising: a thermal sensor configured to generate alow frequency and/or direct current signal upon receiving thermalenergy; a spatially modulating optic disposed between the thermal sensorand the warm object configured to modulate the thermal energy receivedby the thermal sensor as a function of an orientation of the warm objectwith respect to the thermal sensor; and an electronics unit incommunication with the thermal sensor, further comprising: a memory; anda processor in communication with the memory, the processor configuredby the memory to perform steps comprising: detecting a change in thethermal sensor signal; and recognizing a characteristic in the thermalsensor signal.
 2. The system of claim 1, wherein the thermal sensor isconfigured to detect radiation having a wavelength at least within therange of 4 μm and 20 μm.
 3. The system of claim 1, wherein the thermalsensor is one of the group consisting of a thermopile, amicroelectromechanical (MEMS) infrared sensor, a pyroelectric sensor, abolometer, an intrinsic infrared semiconductor, and an infraredextrinsic semiconductor.
 4. The system of claim 1, wherein the spatiallymodulating optic comprises one of the group consisting of a ground ormolded multi-lens array, a molded Fresnel-lens array, and a combinationof a multi-lens and a Fresnel-lens array.
 5. The system of claim 1,wherein the spatially modulating optic further comprises one or more ofthe group consisting of a spatial aperture array with total or partiallight exclusion between apertures, a grating, a coding plate or disc. 6.The system of claim 1, wherein the characteristic is detected in thetime domain.
 7. The system of claim 1, wherein the processor is furtherconfigured by the memory to perform steps comprising converting thethermal sensor signal from a time domain signal to a frequency domainsignal.
 8. The system of claim 7, wherein the characteristic is detectedin the frequency domain.
 9. The system of claim 7, wherein thecharacteristic comprises a correlation between a frequency domain eventand a time domain event.
 10. The system of claim 1, wherein the thermalsensor and the electronics unit are co-located within a singleapparatus.
 11. The system of claim 1, wherein the thermal sensor, thespatially modulating optic, and the electronics unit are co-locatedwithin a single apparatus.
 12. The system of claim 1, wherein thethermal sensor and the spatially modulating optic unit are co-locatedwithin a single apparatus.
 13. A method for recognizing a gesture of awarm object moving in a monitored space, comprising the steps of:receiving incident thermal energy at a modulating optics from a field ofview of the modulating optics within the monitored space, wherein themodulating optics comprises a plurality of lenses and/or apertures;directing the incident thermal energy received by the modulating opticsonto a thermal sensing device optically coupled to the modulatingoptics; producing, with the thermal sensing device, a direct currentoutput signal that is sustained at a level proportional to an amount ofthermal energy being directed to the thermal sensing device by themodulating optics; and providing the output signal to an electronicsunit in communication with the thermal sensing device, the electronicsunit further comprising a memory and a processor in communication withthe memory, the processor configured by the memory to perform stepscomprising: isolating a characteristic of the signal; and comparing thecharacteristic of the signal to a reference characteristic.
 14. Themethod of claim 13, wherein the modulating optics provides optical zoneswith varying output signal over the individual zone thus producingalternating regions of relatively high output signal to the incidentthermal energy and regions of relatively low output signal to theincident thermal energy.
 15. The method of claim 13, wherein the thermalsensing device is selected from the group consisting of amicro-electro-mechanical (MEMs) infrared sensor, a thermopile, abolometer, pyroelectric and a semiconductor-based infrared sensor. 16.The method of claim 13, wherein the lenses of the modulating optics areselected from the group consisting of Fresnel lens arrays, Fresnel zonearrays, holographic optical elements, diffractive optical elements,refractive optical elements and binary optical elements.
 17. The methodof claim 13, wherein the spatially modulating optic further comprisesone or more of the group consisting of a spatial aperture array withtotal or partial light exclusion between apertures, a grating, a codingplate or disc.
 18. The method of claim 13, wherein the direct currentsignal output is sustained at a level that is proportional to an amountof thermal energy being directed to the thermal sensing device with awavelength between 4 μm and 20 μm.